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Abstract 

We show that Ao's stochastic process [J. Phys. A: Math. Gen. 37, L25 (2004)] is 
consistent with a phase- volume preserving conservative dynamical system in equilib- 
rium with a stochastic damping satisfying detailed balance: dq = (7 — DV<p)dt + 
d£(t) where V • 7 = 0, (£(*)£(*')) = 2D6(t - t'). The zeroth law of thermody- 
namics implies stationary density u ss (q) = e~^ q \ We show that a hallmark of such 
systems is an orthogonality between the gradient of the stationary density and the 
conservative rotation in phase space: V(f> -7 = 0. A stochastic thermodynamics 
based on time reversal (£,7) — > ( — t, — 7) yields a novel formula for heat dissipation 
h* d (t) = - J Rn V(p-DX7 In (u(q, t)e^)dq and entropy production e*(t) = -dF(t)/dt: 
The "free energy" F(t) never increases. Entropy balance equation ^ = e* — h* d is 
also recovered. The relations among Ao's stochastic process, a Hodge-like potential- 
flux decomposition, stochastic Hamiltonian system with Klein- Kramers equation, and 
the theory of GENERIC, are discussed. When <fi is given, three different types of time 
reversal leading to different open-system thermodynamics are suggested. 

1 Introduction 

Significant progress has been made in recent years on stochastic dynamics and thermo- 
dynamics of mesoscopic systems. See E 12 El SJ and references cited within. Histori- 
cally in chemistry and life sciences, one of the major motivations of stochastic dynamics 
is overdamped single macromolecule in aqueous solution — the polymer theory [51 [6J. 
While overdamped stochastic dynamics is extensively studied with wide applications, un- 
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derdamped systems with stochastic damping has attracted less attention EKH, partly be- 
cause the singular nature of the diffusion process represented by Klein-Kramers equation, 
the Kolmogorov forward equation for a stochastic Hamiltonian dynamics [171191 [TOlfTTTl FI 

In the theory of dynamical systems, conservative dynamics with volume-preserving 
flow in phase space is the natural generalization of Hamiltonian dynamics [fT3l . Ordinary 
differential equation q = 7(g) with V • 7(g) = can also be represented as flow in phase 
space du t (q,t) = —7(g) • Vu(q,t). When coupled to an stochastic, inert damping, it is 
natural to consider a stochastic differential equation (SDE) 

dq{t) = j(q)dt + { - DV(j)(q)dt + d£(t)\ (1) 

in which the terms inside {■•■}, in the spirit of P. Langevin |fl4ll . represent the friction 
and collision of a stochastic damping: (£(£)£(£')) = 2DS(t — t'). With a non-driven, inert 
stochastic damping, the stationary density of the system (OQ), u ss (q), has to be the same as 
the stationary density of the damping alone: 

u ss (q) = e~ 0(9) . (2) 

This is the zeroth law of thermodynamics; it is also the spirit of detailed balance in sta- 
tistical chemistry and fluctuation-dissipation relation in statistical physics: If a bath is in 
equilibrium with a system, then the equilibrium bath is invariant irrespective of whether it 
is in contact with the system or not. A nonequilibirum steady state arises when the con- 
dition in © is violated due to a dis-equilibrium between the bath and the conservative 
dynamical system |[8l[T2l[T5l[T6l[T71[l8]|. Such a stochastic bath contains active energy as 
driving force lfT9l . 

In 2004, Ao has proposed a novel form of a stochastic process together with a decompo- 
sition scheme that yields stationary probability density and stationary rotational motion, as 
well as a steady state thermodynamics [I201I2T1I . This novel stochastic theory has intrigued 
and mystified many researchers. Ao and his coworkers have also carried out explicit com- 
putations for linear stochatsic differential equations following his general theory |[22l . One 
of the key features in the linear system is an orthogonality between the gradient of station- 
ary density and the stationary probability flux. 

1 1 thank Prof. Min Qian of Peking University for extensive discussions on stochastic processes, dynamical 
systems, and time reversibility. His 1979 paper ifTTTl in Chinese, which had inspired lfT2ll . has attemped to 
introduce a stochastic Hamiltonian system along the line of Klein, Kramers, Wang and Uhlenbeck, and 
illustrated the importance of fluctuation-dissipation relation in obtaining Gibbs' canonical ensemble. 
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In the present work, we show that Ao's stochastic processes [|20ll is indeed a stochas- 
tic dynamics with volume-preserving flow in contact with a detailed-balanced stochastic 
dampling, i.e., Eqs. [T] and |2j We shall also clarify the foundation of Ao's conservative 
thermodynamics ll2TTl : It is based on the time reversal (t, 7) — > ( — t, —7) for Eq. [T1 [|23l . 
Under this time reversal, entropy production is in fact the decreasing rate of a free energy 
functional -dF/dt, where flZQ 

F[u(q,t)}= [ u(q,t)ln(u(q,t)e^)dq. (3) 

—dF/dt, which is non-negative, has been called free energy dissipation in ll24l and non- 
adiabatic entropy production in |[25l |26l |T7J . 

2 Conservative dynamics with stochastic damping 

2.1 Ao's stochastic process 

Following ||20lr s notations, one has 

[£(?) + A(g)]^ = -V#g) + C(ff,t)> ( 4 ) 

where 

(aq,t)( T ( q ,t')} = 2S(q)5(t-t'). (5) 

Ao further introduced a transformation via an auxiliary matrix inversion (S(q) + A(q)) 1 
= G(q) and obtained GUI 

^ = -G(?W(g)+£(g,t), (6) 

with 

({;(q,t)f(q,t')) = (G(q)((q,t)( T (q,t')G(qf 

= 2G(q)S(q)G T (q)5(t-t>). (7) 

Then the associated Kolmogorov partial differential equation in divergence form, following 
GHl[2l,is, 



dt 



G(q)S{q)G T (q)Vu + G{q)V<P{q)u 



(8) 



3 



One of the key results in [|20| is that the stationary density for the stochastic process is 
e -0(<j) Then the stationary flux J(q) satisfies 11301 



J(q)e+M = G(q)S(q)G T (q)V ( j>(q) - G(q)V(j>(q). 



(9) 



From Eq. [9] one has 

(V0(g)) T - J{q)e*® 



(V0(g)) • [GSG T V0(g) - GV0(g) 

T 



G 1 V0(g) 



G T V0(g) 



S + G- T 
S + S + A T 



G 1 V0(g) 



G T V<P{q) 



0. 



(10) 



Since ^ 0, Eq. [lOl means (V0(g)) T • J(g) = is a necessary condition for Ao's 
stochastic processes. 

Wfang's Hodge-like decomposition. The orthogonality in Eq. [10] leads to several inter- 
esting properties for a stochastic dynamics. First, noting the stochastic differential equation 
(SDE) in ©, dq(t) = b(q)dt + d£(t), and the relation in ©, we have the drift 



b(q) = -G(q)V<t>(q) = -DVftq) + J(q)e««\ 
and (£(*)£(*')> = 2£>5(t - t'). Denoting 7 (g) = J{q)e^\ then 

b(q) = -DV4> + 7 , V • 7 = 0, V0 ■ 7 = 0. 



(11) 



(12) 



The right-hand-side of (fTTI) are Wang's potential and flux landscapes [[301 for general SDE. 
The orthogonality between the gradient and curl terms is an additional feature of Ao's 
stochastic processes. The first two equations in (PT21) are a Helmholtz-Hodge-like decom- 
position with diffusion matrix D PTfl . As far as we know, there is no orthogonality in a 
Hodge decomposition in general. 

Xing's Hamiltonian system. Such an orthogonality is a necessary consequence for a 
damped Hamiltonian system (8KIT1 in equilibrium with detailed-balanced stochastic damp- 
ing: 

d ( x\ ( ^ + Vx(x,y) 



dt 



dt 



y 



dH 

dx 



r(x,y) 



(13) 



dt 
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Its generalized Klein-Kramers equation 



9u /_ _ X 

^ = (v„v y ) 



v„ 



F(x,y)T T (x,y) 



u 



(14) 



has a reversibility condition for the damping and the noise H32H : There exists an equilibrium 
between the system and the heat bath: u eq = e H such that 



T(x,y)T T (x,y) 



Vx(x,y) 



u eq (x,y) = 0. 



(15) 



Then the stationary distribution to (1141) with both Hamiltonian and heat bath parts is again 
u eq = e~ H with rotations in phase space: 



J(x,y) = - 



dH 
dy 

dH 

dx 



u eq (x,y). 



(16) 



Indeed, Vu eq ■ J = 0. Eqs. (|T4|) and (1151) are based on Ito's integration. It is easy to 
see that if another convention for integration is chosen, both equations will have different 
expressions; but Eq. (fT6l) is invariant. 

Eqs. CCD, CC§' and (HU) together suggest that for any SDE dq(t) = b(q)dt + d£(t) with 
stationary density e^* and flux J, if V0 ■ J = 0, then the SDE can be re-written as 



dq(t) 



Dv<f>( q ) + dm\ + i(q) 



(17) 



in which the first two terms in the {■ ■ ■ } is from the heat bath with detailed balance. They 
represent the "friction" and "collision", i.e., dissipation and fluctuation. The dynamics 
described by the 7(g) is a deterministic conservative system |fT3l with a volume preserving 
flow in phase space: 



4 I d( l 



dt 



7(g) • dS 



V • 7(g) dq = 0. 



(18) 
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Ao's process indeed has been mapped it to a Hamiltonian system while the stochastic 
damping corresponding to a harmonic bath [33]. The relationship between this Hamil- 
tonian which includes a large harmonic bath and another one emerging from path-integral 
formalism 113411351 remains to be elucidated. 

Grmela-Ottinger's GENERIC form. If the 7(g) in Eq. [T] is a Hamiltonian system, 
then the deterministic part of the system has the GENERIC form proposed by Grmela and 
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Ottinger |[36il: 



dx 
dy 



dH 

dy 

dH 

dx 



D xx D 



&yx -^yy 




dt + 




(19) 



The orthogonality has also figured prominently in the GENERIC structure which has a rich 
geometric interpretation. 

A generalized Klein-Kramers equation. As we have already noted, while the choice of 
the intergration for an SDE with multiplicative noise is not unique, e.g., Ito, Stratonovich, 
or Ao's divergence form [29], there is no ambiguity at the partial differential equation level. 
A conservative dynamics in equilibrium with stochastic damping can be written as, in Ito's 
form: 



du(q, t) 
dt 



V(D(q)u(q,t)) - T](q)u(q,t)) - j(q)u(q,t) 



with r](q) and 4>(q) are related via 

v(D{q)e-^ q) 



rj{q)e 



-Hi) 



0. 



(20) 



(21) 



Solving rj(q) from Eq. (l2T|) and substituting it into (l20l) . we have 

du(q,t) v 



dt 



D(q) Vu(q, t) + u{q, t)V(j){q) - l{q)u{q, t) 



(22) 



which is precisely the divergence form Ao advocated. Eqs. [14] and [15] together are a special 
case of ([22]). In fact, £[5]) implies that (Q3]) has the form 

V x In u eq 
V y In u eq 





2.2 Stochastic thermodynamics 

A volume-perserving conservative dynamics q = 7(g) is time-reversible in the sense t — > 
—t and 7 — > —7 [fT3l . In terms of such a time reversal, let us consider a stochatsic path 
u t = {?( s )|0 < s < t} and its time-reversed path cu^ = {q(t — s) 1 < s < t} under the 
probability measures P +7 and P -7 defined by SDEs dq = (7 — DV(p)dt + d£(t) and its 
time-reversal dq~ = ( — 7 — DV(f)dt + d£(t), respectively. A sample-path based entropy 
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production can be introduced Il32ll37ll38ll39ll40ll: 



t+7 



In 



(23) 



jm - v K^)^ (24) 

u(g, t)V In (u(g, t)e^) ■ £>V In (u(g, t)e^ q Adq > 0, (25) 

where J(q,t) = (7 — DV<f))u(q, t) — DVu(q, t), and i?^ 7 [ • • • ] denotes ensemble everage 
with respect to P +7 . We note that the 7 term has disappeared in the final expression: 
Entropy production is a purely determined by the damping mechanism. 

It is important to point out that the e* introdiced in Eq. [23]is different from the standard 
entropy production for a stochastic diffusion 11321 [381 [39l |40l . In the literature, e* has been 
called as free energy dissipation or non-adiabatic entropy production [|24ll25l[26ll27l . 

In terms of the e* one can define a nonequilibrium free energy flOl . 



F(t) = f^ M ^M^ dq . (26) 
Then its time change, as has been shown in j[2T| : 

^ = ^K^)^-^- ' <27) 

Eq. [27] should be interpreted as follows: For Gibbs' canonical ensemble with non-uniform 
u eq (q) = e~^ q \ the thermodynamic potential is free energy F whose decreasing rate is the 
entropy production e*. 

We also note that the nonequilibrium free energy F(t) = U(t) — S(t) with 

U{t)=- I u(q,t)\nu eq (q) dq, S(t) = - [ u{q,t)\nu{q,t) dq. (28) 

They can be interpreted as internal (conservative) energy and entropy, respectively. Fur- 
thermore, 

7- ( / u(q,t)\nu eq (q)dq) = - f V0 ■ DV In (u(q, t)e+) dq. (29) 

We note again that the 7 term has completely disappeared. The right-hand-side can be 
interpreted as pure heat dissipation, h* d (t). Then, 

^ = e* p (t) - h* d (t), h* d = - J V0 • DV In (u(q, t)e*) dq, (30) 



which is known as the entropy balance equation in Dutch school of nonequilibrium ther- 
modynamics IBTII : ^ = + %r> See also [8J for a discussion on the meanings of these 
terms as the system's entropy change, total entropy prodction, and heat dissipation. 

2.3 Three different time reversals 

We now consider a general SDE. In Roll , we have introduced the notion of a canonical 
conservative dynamics with respect to a differentiable invariant density p(x): x = j(x) 
with V ■ (p(x)j(x)) = 0. A general SDE dx = b(x)dt + d£(t), when its stationary density 
is known, can always be re-written as ll40ll 



dx(t) = (b{x) + D(x)V(f>{x))dt + { - D[x)V<j>(x) dt + d£(t)}, (31) 

in which = u ss {x) is the stationary density. Then j(x) = b(x) + D(x)V \n<f)(x) is 

a canonical conservative dynamics with respect to u ss (x). It has been shown in HOl that 



under time revera 



(t,j) —>•( — £, — j), the system in (1311) has again entropy production 

e*(t) = -dF/dtl' 

One can further decompose the term j(x) into parallel and perpendicular to V0(x): 
j(x) = j\\(x) +j±(x), with 



n (x) = ( '^^{f ) V0(x), n{x)=j{x) - n (x), (32) 
where V0(x) • j±(x) = 0. Then 

b(x) = -D(x)V(f)(x) +]±(x)+ n (x). (33) 
The last term can also be written as 

It represents the non-conservative nature of j(x) ll44l . According to time reversal (t, j) — > 
( — t,j), all j(x) contributes to stationary entropy production [32l [3j ; and according to 
time reversal (t,j) — > ( — t, — j), there will be no stationary entropy production li40l . The 
present work and Eq. [33] suggests yet another time reversal: (t, j±) —>•( — £, —j±), under 



2 It is intriguing to note that according to D. Ruelle, for a smooth dynamical systems (e.g., Anosov diffeo- 
morphisms), entropy production is always zero if there is an invariant density |42l : It only becomes strictly 
positive when an invariant measure is singular, i.e, fractal 1431 . 
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which stationary dissipation arises from j». These three different time reversals reflect 
assumptions based on over-damped, non-damped or under-damped nature of a stationary 
dynamics. 



3 Discussion 

3.1 Orthogonality for infinitesimal noise 

We now consider a general stochastic differential equation with an infinitesimal stochastic 
term: 

dq(t) = b{q)dt + y/ed£(t), = 2D $(t ~ 0, (35) 

with Fokker-Planck equation 

= _ V ■ J(q, t), J(q, t) = b(q)u(q, t) - eDVu{q, t). (36) 
The large deviation principle suggests that 113511 

u ss (q) = exp , (37) 

and thus, J ss (q) = u ss (q) (b(q) + DVip e {q)) . Then V ■ J ss (q) = implies 

eV • (b(q) + DVMQ)) = (&(?) + DV^{q)) ■ W> e (?)- (38) 

Reversibility under time reversal (t, 7) — > ( — t, 7) is eqivalent to 7 = b + D\7ip e = 
|[32][3l|4||. hi the present work with time reversal (t, 7) — > ( — t, —7) , 7 7^ but V ■ 7 = 
7 • = 0. One could consider these conditions as successive "solvability conditions" 
for the steady state of a stochastic dynamics. 

Eq. [38]is exact. We now consider the limit of small e, and ip e (q) — 4>o(q) + ^4>i (?) H > 

where 

V0 o (g)- ODWo(g) + &(?)) =0, (39) 

V^i(g) • (2 J DV0 o (g) + 6(9)) = V ■ (£V0o(<?) + &(?)), ( 4 0) 

vMq)- (2dwMq) + Hq)) = v ■ (dv&Oz)) - (v<M?))£(v<M<?)). (4i) 

We thus have a decomposition that is resemblant to that of Hodge: 

b{q) = -DVMq) + (&(?) + DVMvj) , (42) 



in which the two terms are orthogonal by the definition of <fio(q), as given in ( |39l) . However, 
Eq. [38] indicates that, in the limit of e ->■ 0, V • (b(q) + DV<f) (q)) is g. In fact, from (EH 

V • (b(q) + DV0 o (<?)) = -V0i(g) ■ (2 J DV0 o (g) + 6(g)) (43) 

is on the order 0(1). Only when 4>\{q) = a constant, the equation in (1421) becomes an or- 
thogonal Hodge decomposition in the asymptotic limit of e — > 0. Ao's stochastic processes 
has all (f)i(q) = with i > 1. 

3.2 Boltzmann's thermodnamic probability and Kolmogorov back- 
ward equation 

Motivated by the Eq. [2H let us introduce tl(q, t) = u(q, t)e^ q \ Then 

= V • DVQ(q, t) - ( 7 (g) + DV^g)) ■ Vfi(g, t). (44) 
The Kolmogorov backward equation corresponding to (l22l) is 

= V ■ DVv(q, t) - (DV<t*(q) - 7 (g)) ■ Vv(q, t). (45) 

They differ only by a 7 — >• —7. The function £l(q,t) can be interpreted as Boltzmann's 
"thermodynamic probability", whose logarithm is a thermodynamic potential (Boltzmann 
entropy). Finally, the probability flux consists of a "mechanical" force 7 and an "entropic" 
force -DVlnfi: 

J(g,*)=7(9)-£Vlnfi(g,*). (46) 

We also note that for both Eqs. |44]and|45l if v(q, t) > 0, there is an Boltzmann's H-theorem 
like relation: 

j t j (p(q 7 t)hin(q,tj)e-+®dq = ~J (™) ■ (DVQ) f^j dq < 0. (47) 

3.3 Conservative and dissipative dynamics 

The notions of conserative and dissipative systems are fundamental concepts in mechanics 
and thermodynamics. They are at the core of I. Prigogine's challenge to Newton-Laplace's 
world view B5l l46l l47ll . Entropy production is the key mathematical quantity charac- 
terizing dissipation. Since 1980s, it has become increasingly clear that the mathematical 
foundation of entropy production lies at the notion of time reversal 11321 1371 1381 . 
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For the dynamics associated with system in (OQ), classical Newtonian notion of time 
reversal is (t, 7) — > ( — t, —7) which gives rise to an entropy production e* solely by the 
non-adiabatic entropy production —dF/dt. On the other hand, if one chooses time reversal 
(t, 7) — > ( — t, 7) , then total entropy production e p is the sum of both adiabatic and non- 
adiabatic entropy productions Il24l |27l . The adiabatic entropy production is also known 
as house-keeping heat [|48l : It represents the amount of active energy input to sustain a 
nonequilibrium steady state [|40l . 

To put the above discussion into sharper contrast, consider the following biophysical 
experiment on a single motor protein in the presence of given ATP and ADP concentrations 
in solution. At 100 cycles per second, a motor protein runs for a day with a total ~ 10 7 
ATP hydrolysis. However, at a millimolar concentration in a millilitre volume, there are 
10 17 total number of ATP molecules. Hence, with a single motor protein running for a day 
in such a mesoscopic system, the concentration of ATP changes only 1 part of 10 billion. 
It is essentially undetectable. 

Now consider an experiment on a type II superconducting ring with a current in the 
presence of a magnet. This system has been considered in condensed matter physics as 
an equilibrium system. However, according to Newton's third law, the supercurrent has 
to exert a force on the magnet and causing it to slowly demagnetize even though it is 
essentially undetectable l|49l . 

Newton's first law states that a linear constant motion persists in the absence of a force. 
A rotatinal motion, hence, requires a force, even when it does no work. According to 
Newton's third law and our understanding of the constituents of matter, there is always a 
consequence, a la Lord Kelvin, at the origin of the force that causes a rotational motion. 
The environment of an open system, therefore, can not be time reversible. 

Conservative or dissipative description of an open system are mathematical models 
which depend upon an experimentalist's knowledge and perspective. They are rather theo- 
retical issues; the dynamics is closer to the realityjj 

I thank Ping Ao, Hao Ge, Da-Quan Jiang, Masaki Sasai, Jin Wang, and Jianhua Xing 
for many helpful discussions. 

3 I thank Prof. Madeleine Dong (University of Washington) for continuous discussions on the nature of 
interpretations, narratives, and discourse in historical research. 
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